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(54) Electronic watermark system 

(57) An electronic watermark system invisibly 
embeds watermark information into original image data 
and the watermarked image data is transferred to a first 
medium (504, 604). At the same time, the embedded 
watermark information is transferred to a second 
medium (505, 605). When a medium questioned is 



found, its watermark information can be easily identified 
by searching the second medium. Alternatively, the 
watermarked image data and the embedded watermark 
information are transferred to different storage areas 
(705, 706) of the same medium (704), respectively. 
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Description 



The present invention generally relates to an 
authorization system, and in particular to an authoriza- 
tion system using identification information embedded s 
into image data to prevent unauthorized duplication of 
the image data. 

Recently, unauthorized copying of digital image 
data has become a serious problem because duplica- 
tion of digital data can be extremely easy. To prevent w 
unauthorized copying, several authorization systems 
using identification information (an electronic water- 
mark) embedded into image data have been proposed 

A visible-watermarking system is disclosed in Japa- 
nese Patent Unexamined Publication No. 8-241403 is 
which corresponds to United States Patent No. 
5,530,759. In this system, a visible watermark is placed 
on a digital image such that the corresponding pixel of 
original image changes its brightness but its chromatic- 
ities. 

A . . ... 20 
An invisible-watermarking system is disclosed in 
NIKKEI ELECTRONICS 1996.4.22 (no. 660). In this 
system, original image data is converted into frequency 
spectrum; and then ID information is embedded into the - 
frequency spectrum; which is in turn converted into. 25 
image data invisibly including the ID information. In 
authorization check, image data questioned is con- 
verted into frequency spectrum. ID information embed- 
ded in the image data questioned is obtained from a' 
difference between the frequency spectrum questioned'* 30 
and the original frequency spectrum. Since the ID infor- 
mation is embedded into frequency spectrum of the 
original image data, it has little effect on the image qual- 
ity and further it has become very difficult to delete the 
ID information from the image data. 35 

In the above invisible-watermarking system, since 
the ID information cannot be seen, it is difficult to check 
whether image data is authorized in the case where no 
one knows the ID information. Especially, in a distribu- 
tion system of data storing medium including CD- 40 
ROMs, there are cases where a different watermark is 
used for each distribution channel. In this case, the 
authorization check and manufacture management 
become very complicated and time-consuming jobs. 

All object of the present invention is to provide a 4s 
system which can rapidly check whether image data is 
authorized to effectively prevent unauthorized copying 
of the image data. 

Another object of the present invention is to provide 
a system which can easily identify an electronic water- so 
mark invisibly embedded in image data to check 
whether the image data is authorized. 

According to an aspect of the present invention 
watermarked image data having watermark information 
irwisibly embedded is transferred to a first medium and 55 
the embedded watermark information is transferred to a 
second medium. Since the embedded watermark infor- 
mation is stored in the second medium, watermark 



identification can be easily made. 

Further, in an authorization check system for check- 
ing whether a medium questioned is authorized, a stor- 
age stores watermark information and medium 
identification information identifying a medium, the 
medium storing watermarked image data which is pro- 
duced by invisibly embedding the watermark informa- 
tion into original image data. A retrieval controller 
retrieves watermark information corresponding to 
medium identification Information of the medium ques- 
tioned from the storage. A watermark extractor extracts 
watermark information questioned from image data 
stored in the medium questioned. A checker checks 
whether the medium questioned is authorized by com- 
paring the extracted watermark information with the 
retrieved watermark information. 

According to another aspect of the present inven- 
tion, watermarked image data having watermark infor- 
mation invisibly embedded and the embedded 
watermark information are both transferred to a single 
medium. Preferably, the embedded watermark informa- 
tion may be. stored in a dedicated storage area of the 
single medium: Further, the embedded watermark infer- * 
mation may be encrypted and stored. Since the embed; 
ded watermark information is stored in the same 
medium/watermark identification can be easily-made. 

Further , in an authorization check system for check- 
ing whether a medium questioned is authorized the 
medium questioned having a first storage area for stor- 
ing watermarked image data and a second storage area- 
for storing watermark information. After extracting 
watermark information from image data stored in the 
medium questioned, a checker checks whether the 
medium questioned is authorized by comparing the 
extracted watermark information with the watermark 
information stored in the second storage area of the 
medium questioned. 

Fig. 1 is a schematic block diagram showing an 
encoding side of an electronic watermark system 
according to the present invention; 

Fig. 2 is a schematic block diagram showing a 
watermark check section of the electronic water- 
mark system according to the present invention; 

Fig. 3 is a schematic diagram showing an example 
of watermarking section of the encoding side of Fig. 

Fig. 4 is a schematic diagram showing an example 
of a watermark extractor of the watermark check 
section of Fig. 2; 

Fig. 5 is a diagram showing a first embodiment of 
an electronic watermark system according to the 
present invention; 
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Fig. 6 is a diagram showing an electronic water- 
mark system according to a second embodiment of 
the present invention; 

Fig. 7A is a schematic diagram showing an encod- 
ing side of an electronic watermark system accord- 
ing to a third embodiment of the present invention; 
and 

Fig. 7B is a schematic diagram showing a decoding 
side of the electronic watermark system according 
to the third embodiment. 

ENCODING SIDE 



Referring to Fig. 1 , an encoding side of an elec- 
tronic watermark system inputs a stream of original 
image data D 0 and produces both MPEG (Motion Pic- 
ture Experts Group) data stream D 4 and watermark 
data which is invisibly embedded into the MPEG data 
D 4 . More specifically, a stream input section 101 inputs 
a stream of the original image data which is converted 
to frequency spectrum data D, by a DCT (Discrete 
Cosine Transform) section 102. A watermarking section 
103 reads a watermark D WM selected from a plurality of 
watermarks stored in a watermark data table 104, and 
then embeds the selected watermark D WM into the fre- 
quency spectrum data D A to produce watermark- 
embedded frequency spectrum data D 2 . The water- 
mark-embedded frequency spectrum data D 2 is quan- 
tized by a quantizing section 105 and the quantized 
data D 3 is encoded to produce MPEG data by an 
encoding section 106. A transfer controller 107 inputs 
the MPEG data from the encoding section 106 to pro- 
duce a stream of the MPEG data D 4 and a transfer con- 
troller 108 inputs the selected watermark D WM from the 
watermark data table 104 to produce selected water- 
mark data. The stream of the MPEG data D 4 is trans- 
ferred to a storage medium or a client through a 
communication channel and the selected watermark 
data is transferred to another storage medium or the 
same storage medium, as will be described later. 

It should be noted that the above sections 101-106 
may be implemented with a program-controlled proces- 
sor such as CPU or DSP. In other words, the processor 
runs programs including the functions of the above sec- 
tion 101-106 to perform the watermarking and MPEG 
data compression as described above. 

WATERMARK CHECKING SECTION 

Referring to Fig. 2, in a watermark checking section 
of the electronic watermark system, a stream data input 
section 201 inputs a stream of MPEG data D 4 ' and a 
watermark input section 202 inputs watermark data 
associated with the input MPEG data D 4 \ The input 
MPEG data D 4 * is decoded to produce date D 3 ' by a 
decoding section 203 and then the date D 3 ' is inverse- 



quantized to produce data D 2 ' by an inverse-quantizing 
section 204. On the other hand, the watermark checking 
section is provided with an original image memory 205 
storing the original image data D 0 . The original image 
5 date D 0 is converted to frequency spectrum data D A by 
a DCT section 206. 

When receiving the input frequency spectrum data 
D 2 ' from the DCT section 204 and the original fre- 
quency spectrum data from the DCT section 206. a 
10 watermark extractor 207 extracts a watermark D WM1 
from the input frequency spectrum data D 2 * by calculat- 
ing a difference between the input frequency spectrum 
data D 2 * and the original frequency spectrum data 
An inner product calculating section 208 inputs the 
is extracted watermark D WM1 from the watermark extrac- 
tor 207 and the received watermark Dw M2 * rom the 
watermark input section 202. and performs the inner 
product thereof to produce a degree of statistical simi- 
larity between them. An authorization check section 209 
20 checks whether the extracted watermark D WM1 is iden- 
tical to the received watermark D WM2 by comparing the 
degree of statistical similarity with a reference value. 

The input MPEG data D 4 may be received from a 
storage medium or a communication network. The 
25 watermark data associated with the input MPEG data 
D 4 is received from a management database storing 
manufacture management data or the same storage 
medium as the MPEG data D 4 \ as will be described 
later. In the case where the input MPEG data D 4 and/or 
30 the watermark data are received from a communication 
network, the watermark checking section is provided 
with a communication means such as a network inter- 
face or a radio transceiver. 

It should be noted that the above sections' 20 1-209 
35 may be implemented with a program-controlled proces- 
sor such as CPU or DSP. In other words, the processor 
runs programs including the functions of the above sec- 
tion 201-209 to perform the watermark extracting. 
MPEG data decompression and authorization check as 
40 described above. 



WATERMARKING 

Referring to Fig. 3. the watermarking section 103 
45 receives the frequency spectrum data from DCT 
section 102 and selects a set of N data samples: f(1)- 
f(n) which are greater than a predetermined threshold 
level from the frequency spectrum data D v Further, the 
watermarking section 103 selects a set of watermark 
so data: w(1)-w(n) from random numbers depending on a 
normal distribution with a mean of 0 and a variance of 1 . 
The watermarking section 103 calculates 
F(i) = f(i) + a|f(i)| # w(') for each variable i 0 5 i S n) 
using multipliers 301 and 302. a constant a and an 
55 adder 303. where a is a scaling element (hereinafter, 
assuming a = 1). The calculated data samples F(1)-F(n) 
are substituted for the selected N data samples f(1)-f(n) 
of the frequency spectrum data D<\ to produce water- 
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marked DCT frequency spectrum data which will be 
subject to inverse DCT in the following stage 1 05. In this 
manner, the selected watermark D WM is invisibly 
embedded into the original image data D 0 . 

WATERMARK CHECK 

Referring to Fig. 4, the watermark extractor 207 
inputs the received data samples F(l)-F(n) of the 
received frequency spectrum data D 2 ' and the original 
data samples f(l}-f(n) of the original frequency spec- 
trum data D v The watermark extractor 207 calculates 
w i(') = (FO) -f(i))/f(i) for each variable i (1 = i * n ) 
using subtracter 401 and a divider 402 to extract the 
watermark D WM1 = ( Wi (1), Wl (2), • • • . Wl (n)). 

Subsequently, the inner product calculating section 
208 calculates a degree of statistical similarity C 
between the extracted watermark D WM1 = ( Wi (1), Wl (2), 
••-.w^n)) and the received watermark 
D WM2 °(w 2 (1), w 2 (2). • • - .w 2 (n)) using the fol- 
lowing equation: C = D WM , ♦ D WM2 / |D WM2 , . 
If C is equal to or greater than the predetermined level, 
it is determined that the received watermark is embed- 
ded into the received MPEG data D 4 and therefore the 
received MPEG data D 4 ' is the authorized data. If C is 
smaller than the predetermined level, it is determined 
that the received MPEG data D 4 * is an unauthorized 
duplication. 



FIRST EMBODIMENT 

Referring to Fig. 5. the encoding side of the elec- 
tronic watermark system is comprised of a processor 
501 , the watermark data table 104. a data storage con- 
troller 502 and a data transfer controller 503. As 
described before, the processor 501 runs programs 
including the functions of the sections 101-106 as 
shown in Fig. 1 to perform the watermarking and MPEG 
data compression. 

The processor 501 inputs a stream of original 
image data D 0 and produces the water-marked MPEG 
data stream D 4 which is stored onto a storage medium 
504 such as CD-ROM or magneto-optic disc by the data 
storage controller 502. At the same time, the data stor- 
age controller 502 outputs a serial number of the stor- 
age medium 504 to the data transfer controller 503. In 
this manner, an authorized medium 504 storing MPEG 
data stream D 4 into which the selected watermark data 
d wm 's embedded is manufactured. If the processor 
501 selects another watermark data, an authorized 
medium 504 storing the same MPEG data stream D 4 
into which a different watermark data is embedded is 
easily manufactured. Needless to say, the data storage 
controller 502 outputs the serial number of the storage 
medium 504 to the data transfer controller 503. 

The data transfer controller 503 inputs the embed- 
ded watermark data D WM from the watermark data table 
104 and then transfers a pair of the embedded water- 



mark data Dw M and the serial number of the storage 
medium 504 to a manufacture management database 
505 provided in a management center. Therefore, the 
watermark for each authorized medium 504 can be eas- 
5 ily identified by searching the manufacture management 
database 505. In the case where the management 
center is located at a distance from the encoding side, 
the data transfer controller 503 may transfer them 
through a local-area network or a radio communication 
10 channel. 

When a suspect medium 506 is found, the authori- 
zation check is performed by an authorization check 
section 507 running the programs including the func- 
tions of the above sections 201-209 as shown in Fig. 2 
is to perform the watermark extracting, MPEG data 
decompression and authorization check. In this case, 
the authorization check section 507 includes data com- 
munication means. 

First of all, the authorization check section 507 
so reads the serial number from the suspect medium 506 
and transmits a retrieval request using the read serial 
number to the manufacture management database 505 ! 
through a network Upon receipt of the retrieval request, 
the manufacture management database 505 is 
25 searched for the corresponding watermark to the serial 
number. If the corresponding watermark is found, the 
retrieved watermark data is sent back to the authoriza- 
tion check section 507. 

Using the retrieved watermark data, the authoriza- 
30 tion check section 507 performs the watermark extract- 
ing and watermark checking operations as described 
before. That is. if the degree of statistical similarity C 
between the extracted watermark and the retrieved 
watermark is equal to or greater than the predetermined 
35 level, it is determined that the medium 506 is one of 
authorized media. If C is smaller than the predeter- 
mined level, it is determined that the medium 506 is an 
unauthorized duplication. 

40 SECOND EMBODIMENT 

Referring to Fig. 6, the encoding side of the elec- 
tronic watermark system is comprised of a processor 
601 , the watermark data table 104, a data communica- 

45 tion controller 602 and a data transfer controller 603. As 
described before, the processor 601 runs programs 
including the functions of the sections 101-106 as 
shown in Fig. 1 to perform the watermarking and MPEG 
data compression. 

50 The processor 601 inputs a stream of original 
image data D 0 and produces the water-marked MPEG 
data stream D 4 which is transmitted to a client 604 by 
the data communication controller 602. Here, it is 
assumed that the water-marked MPEG data is distrib- 

55 uted depending on a data transmission request 
received from the client 604. At the same time, the data 
communication controller 602 outputs the address 
number of the client 604 to the data transfer controller 
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603. In this manner, the MPEG data stream D 4 into 
which the selected watermark data Dwm is embedded is 
transmitted to the authorized client 604. If the data 
transmission request is received from another client, the 
processor 601 selects another watermark data which is 
embedded into the MPEG data The MPEG data stream 
D 4 into which a different watermark data is embedded is 
transmitted to the net client. Needless to say, the data 
storage controller 602 outputs the address number of 
that new client to the data transfer controller 603. 

The data transfer controller 603 inputs the embed- 
ded watermark data Dwm fro™ the watermark data table 
104 and then transfers a pair of the embedded water- 
mark data D WM and the client address number to a 
manufacture management database 605 provided in a 
management center. Therefore, the watermark for each 
authorized client 604 can be easily identified by search- 
ing the manufacture management database 605. In the 
case where the management center is located at a dis- 
tance from the encoding side, the data transfer control- 
ler 603 may transfer them through a local-area network 
or a radio communication channel. Further the manage- 
ment database 605 is provided with data communica- 
tion controller 606. 

When a suspect data provider 607 is found, the 
authorization check is performed by an authorization 
check section 608 running the programs including the 
functions of the above sections 201-209 as shown in 
Fig. 2 to perform the watermark extracting. MPEG data 
decompression and authorization check. In this case, 
the authorization check section 508 includes data com- 
munication means. 

First of all, the authorization check section 608 
receives the address number from the suspect data pro- 
vider 607 and transmits a retrieval request using the 
address number to the manufacture management data- 
base 605 through the data communication controller 
606. Upon receipt of the retrieval request, the manufac- 
ture management database 605 is searched for the cor- 
responding watermark to the address number. If the 
corresponding watermark is found, the retrieved water- 
mark data is sent back to the authorization check sec- 
tion 608. 

Using the retrieved watermark data, the authoriza- 
tion check section 608 performs the watermark extract- 
ing and watermark checking operations as described 
before. That is. H the degree of statistical similarity C 
between the extracted watermark and the retrieved 
watermark is equal to or greater than the predetermined 
level, it is determined that the data provider 607 is one 
of authorized clients. If C is smaller than the predeter- 
mined level, it is determined that the data provider 607 
distributes an unauthorized duplication. 

THIRD EMBODIMENT 



701. the watermark data table 104. an encryption sec- 
tion 702 and a data storage controller 703. As described 
before, the processor 701 runs programs including the 
functions of the sections 101-106 as shown in Fig. 1 to 
5 perform the watermarking and MPEG data compres- 
sion. Further, the processor 701 may include the 
encryption section 702 and the data storage controller 
703. 

The processor 701 inputs a stream of original 
10 image data D 0 and produces the water-marked MPEG 
data stream D 4 which is stored onto a storage medium 
704 such as CD-ROM or magneto-optic disc by the data 
storage controller 703. At the same time, the encryption 
section 702 encrypts the selected watermark data D^ 
15 and the encrypted watermark data is stored onto the 
same storage medium 704 by the data storage control- 
ler 703. It is preferably that the storage medium 704 has 
a first area 705 for storing the water-marked MPEG data 
and a second area 706 dedicated to the encrypted 
so watermark data. In the case of MPEG data reading 
mode, no data can read from the second area 706. 

In this manner, an authorized medium 704 storing 
the encrypted watermark data D W m and tne water- 
marked MPEG data stream D 4 is manufactured. If the 
25 processor 701 selects another watermark data, an 
authorized medium 704 storing the new watermark data 
and the same MPEG data stream D 4 into which the new 
watermark data is embedded is easily manufactured. 
Referring to Fig. 7B, when a suspect medium 707 is 
30 found, the authorization check is performed by an 
authorization check section 708 running the programs 
including the functions of the above sections 201-209 
as shown in Fig. 2 to perform the watermark extracting. 
MPEG data decompression and authorization check. In 
35 this case, the authorization check section 708 includes 
decryption section 709. 

First off all, the decryption section 709 reads the 
watermark data from the second area dedicated to 
watermark of the suspect medium 707 and decrypts it to 
40 output watermark data Dwm* to the authorization check 
section 708. Using the read watermark data D WM \ the 
authorization check section 708 performs the water- 
mark extracting and watermark checking operations as 
described before. That is, if the degree of statistical sim- 
45 ilarity C between the extracted watermark and the read 
watermark D WM ' is equal to or greater than the prede- 
termined level, it is determined that the medium 707 is 
one of authorized media. If C is smaller than the prede- 
termined level, it is determined that the medium 707 is 
so an unauthorized duplication. 

As described above, since the authorized water- 
mark can be easily obtained from a management center 
or the storage medium storing the watermarked MPEG 
data, the authorization check is rapidly performed with 
55 reliability. 



Referring to Fig. 7A, the encoding side of the elec- 
tronic watermark system is comprised of a processor 
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Claims 

1 . A system comprising 

an information generator (104) for generating 5 
watermark information; and 
a combiner (103) for invisibly embedding the 
watermark information into original image data 
to produce watermarked image data, 

characterized by comprising: io 
a transfer controller (108, 109) for transferring 
the watermarked image data to a first medium 
and the watermark information to a second 
medium. 

75 

2. The system according to claim 1 , wherein the trans- 
fer controller further transfers medium identification 
information identifying the first medium to the sec- 
ond medium to allow retrieval. 

20 

3. The system according to claim 1 or 2, wherein 

the first medium is a first storage medium (504) 
for storing the watermarked image data; and 
the second medium is a second storage 25 
medium (505) for storing the watermark infor- 
mation and medium identification information 
identifying the first storage medium to allow 
retrieval. 

30 

4. The system according to claim 3. wherein the 
medium identification information is a manufacture 
serial number of the first storage medium. 

5. The system according to claim 1 or 2, wherein 35 

the first medium is a communication medium 
through which the watermarked image data is 
transmitted to a destination (604) in a commu- 
nication network; and 40 
the second medium is a storage medium (605) 
for storing the watermark information and 
medium identification information identifying 
the destination to allow retrieval. 

45 

6. The system according to claim 5, wherein the 
medium identification information is an address of 
the destination in the communication network. 

7. A system comprising: 50 

an information generator (104) for generating 
information; and 

a combiner (103) for invisibly embedding the 
watermark information into original image data 55 
to produce watermarked image data, 

characterize by comprising: 
a transfer controller (108, 109) for transferring 



both the watermarked image data, and the 
watermark information to a single medium. 

8. The system according to claim 7, wherein the single 
medium is a storage medium (704) having a first 
storage area (705) for storing the watermarked 
image data and a second storage area (706) for 
storing the watermark information, wherein the sec- 
ond storage area is dedicated to the watermark 
information so that only the watermark information 
is allowed to be read. 

9. The system according to claim 7 or 8, wherein the 
transfer controller encrypts the watermark informa- 
tion and transfers encrypted watermark information 
to the single medium. 

10. A system for checking whether a medium ques- 
tioned is authorized, characterized by comprising: 

a storage (505, 605) for storing watermark 
information and medium identification informa- 
tion identifying a medium, the medium storing 
watermarked image data which is produced by 
invisibly embedding the watermark information : 
into original image data; 
a retrieval controller (505. 605) for retrieving 
watermark information corresponding to 
medium identification information of the 
medium questioned from the storage; 
a watermark extractor (207) for extracting 
watermark information questioned from image 
data stored in the medium questioned; and 
a checker (208, 209) for checking whether the 
medium questioned is authorized by compar-' 
ing the extracted watermark information with 
the retrieved watermark information. 

11. A system for checking whether a medium ques- 
tioned is authorized, the medium questioned having 
a first storage area for storing watermarked image 
data and a second storage area for storing water- 
mark information, the system characterized by 
comprising: 

a watermark extractor (207) for extracting 
watermark information from image data stored 
in the medium questioned; and 
a checker (208, 209) for checking whether the 
medium questioned is authorized by compar- 
ing the extracted watermark information with 
the watermark information stored in the second 
storage area of the medium questioned. 

12. A method for checking whether a medium ques- 
tioned is authorized, comprising the steps of: 

generating watermark information: and 
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invisibly embedding the watermark information 
into original image data to produce water- 
marked image data. 

characterized by the steps of : 
transferring the watermarked image data to an s 
authorized medium; 

transferring the watermark information and 
medium identification information identifying 
the authorized medium to a retrieval medium; 
retrieving watermark information correspond- 10 
ing to medium identification information of the 
medium questioned from the retrieval medium; 
extracting watermark information questioned 
from image data stored in the medium ques- 
tioned: and ,5 
checking whether the medium questioned is 
authorized by comparing the extracted water- 
mark information with the retrieved watermark 
information. 

so 

13. A method for checking whether image data ques- 
tioned is authorized, comprising the steps of: 

generating watermark information; and 
invisibly embedding the watermark information 25 
into original image data to produce water- 
marked image data. 

characterized by the steps of: 
transmitting the watermarked image data to an 
authorized destination through a network; 30 
transferring the watermark information and 
medium identification information identifying 
the authorized destination to a retrieval 
medium; 

retrieving watermark information correspond- 35 
ing to identification information of a data pro- 
vider providing the image data questioned from 
the retrieval medium; 

extracting watermark information from the 
image data questioned received from the data 40 
provider; and 

checking whether the image data questioned is 
authorized by comparing the extracted water- 
mark information with the retrieved watermark 
information. 45 

14. A method for checking whether a medium ques- 
tioned is authorized, comprising the steps of: 

generating watermark information; and so 
invisibly embedding the watermark information 
into original image data to produce water- 
marked image data. 

characterized by the steps of : 
transferring both the watermarked image data 55 
and the watermark information to an author- 
ized medium; 

extracting watermark information from image 



data stored in the medium questioned; and 
checking whether the medium questioned is 
authorized by comparing the extracted water- 
mark information with watermark information 
stored in the medium questioned. 
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